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This memo is to explain why the Artificial Intelligence 
group of Project MAC is developing methods for on-line tracking 
of hu»an eye movements. It also gives a brief resume of results 
to date and the next steps. 



1. 



1, Reasons for developing, real-time cracking systems . 

The study of eye motions is Important already in perceptual 
psychology and in human engineering of ©any kinds of systems. Standard 
methods are based on recording, in one way or another, and analysing the 
results later. These methods do not permit experiments making the visual 
environment conditional on the oculomotor behavior, and are thus limited 
in the extent of experimental control of the stimulus , 

Standard methods of eye-tracking all require either immobilization 
of the subject's head, by clamps or bite-bars, or veering special helmets 
or contact lenses. One of our goals is to allow the subject to operate 
without any special apparatus. Thus, we have to reject special contact 
lenses, which otherwise would make remote tracking fairly easy. 

Now we list reasons for eye-tracking more specifically. 

1.1. Eve-mo tic n s a r c t h c only ob s e rya b 1 £ in t r a - ? c r c op tua 1 p ro c c s s e g . 

The Artificial Intelligence group is concerned with making perceptual 
machinery, and naturally wants to know about human visual mechanisms. 
Unfortunately, too little is known, and this is one instance where it 
appears that we will have to do much of the work ourselves to find out 
what we need. Eye-motions report to us the effect of certain decisions 
in the scene-analysis process; we have no such information in, say, 
audition, (we do, in tactile perception, but the problems here aren't 
so interesting, at least yet). Thus, we ought to be able to find out a 
great deal about the relation between central and peripheral visual field 
cues, by watching eye motions i While this has often been done to some 
extent with off-line recordings, the results of such analyses are only 
interesting, and not very decisive. With on-line information, wc can 
make the visual stimulus— — on a display scope, for example— depend on 
the sequence of fixations in such ways as to make decisive determinations 
of which peripheral cues has what effects. 
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1*2 Clues about Internal thought processes . 

Allen Novell has suggested a number o£ ways that eye-motions might 
be analysed in conjunction with problei^rolving "protocols" so as to help 
trace a subject's execution through his problem-solving flow diagraa. 
Although no really conclusive techniques for this are established, the 
possibilities seem important, for the reasons discussed in 1.4 below. 

1«3 Use as contro_l__systen» input . 

It seems likely that one could use eye-motion for control of complex 
systems, in some very comfortable and precise ways- The program has to 
infer the subjects intentions from his fixations on a possible very complex 
control console, and we can expect serious problems in classifying such 
intentions and recognizing them. Tracking, in particular, should be 
particularly natural. It should be noted that there is no reason to expect 
visual tracking control to be comparable to manual control in precision; 
one must expect to have to need Sketchpad-like inferences about the proximity 
of the fixation from a control point, and from the local behavior of the 
fixation. 

1'^ Hyp o th es i s a bout m smoo th i n t eg rg c ic n wi th memo ry . 

Another reason for supposing that on-line eye-tracking will be important 
is based on the following introspective notion. Normally, one is unconcious 
of details about one's memory of visual scenes, especially when th* scene is 
present- For then, one cat. either remember what is in a portion of the 
visual environment or one can look there. It is possible that these alterna- 
tives are integrated into scene analysis so smoothly that instructions to 
move the fixation point to a point have practically the same status as requests 
to memory for information about what is (supposed to) be there. If something 
like that holds, then we may be able to make dramatically effective and 
theoretically interesting information retrieval systems according to the 
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following schema: Imagine a display with only some very general categories 
of information on it. Assume chat the user is quite familiar with this 
coarse system. When he fixates on a region, this means that he is interested 
in it, and that part of the display is replaced by an expansion in greater 
detail. The subject feels that he knew this material already, in his mind's 
eye, and if he is really familiar with the second-level he will find the 
next sub-category in the expanded portion quickly , again without conscious 
effort. The effect, I hope, would be that subjectively the system will feel 
as though an immense amount of detail is available, in familiar forms, in the 
immediate visual environment, although this is not really true, of course. 
We certainly ought to be able to get simple first-order effects such as 
television pictures that appear to have full bandwidth, though only a soall 
output data rate is really operating. We cannot expect to make the systea 
provide new information In forms that make the subject feel that he already 
"knows" it thoroughly, for we can't expect to get the effect of full networks 
of relations, but one ought to get son* such effects and it seems worth find- 
ing how far it goes. 

2. Simple immediate applications . 

The following experiments could be immensely important in education, 
if they yield good results. 

2.1 Reading without learning to read . 

Suppose that a pre-literate child, adult or blind person (with oculomotor 
control) could look at a page, and have the word or phrase in fixation read to 
him. The eye-tracking program would have to be developed to a point of some 
refinement, of course, to approximate the subject's attention. Then, 
(i) One would expect some learning, 
(ii) One would in fact be able to "read". For the child, this is 
valuable enough so that he would want to keep at it. The control experience 
in reading oneself is different from the experience of being read to. Indeed, 



*. 



it is a special skill in itself, so that 

(iii) the control system for reading can be developed independent 
of the character-and word-reading learning. One need not degrade the 
child back to the "primer" vocabulary. 

2.2 Language learning * 

The probleta of learning to read a foreign lanuage is complicated 
by the need simultaneously to acquire several skills vocabulary, inflec- 
tion, grammatical construction and word-order, pronunciation, and even new 
character sets- Several of these, particularly vocabulary knowledge, can 
be minimized by having the word under fixation persists longer than 
threshold. (Or instead of threshold, some other property of eye-motion, 
or any other subject-output could be used,) Instead of pronunciation, 
one might replace the word by its English equivalent, or if necessary 
the phrase, or one night apply some other transformations. One could guide 
the reader by programmed cues, like brightening, moving, enlarging, delet- 
ing, etc, sections or phrases ahead or behind, to encourage desired scanning 
behavior or other reading skills, 

2.3 Blind . 

Many blind persons have good oculomotor control, or could acquire 
it, given appropriate feedback- Although a blind person can train himself 
to understand text read at higher than normal rates, this is next-to-useless 
for Intricate things like mathematical reading, where one doesn't normally 
scan the page in a "regular" manner. The same is really true for any dif- 
ficult reading, and it is possible that with oculomotor control over a page, 
blind people could absorb the naterial, read-alound fast but locally, at 
very respectable rates, it would be fun to see if anything could be done 
in the line of making aural picture images, with special sounds. 



